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ABSTRACT 

One embodiment of biocybernetic adaptation is a human- 
computer interaction system designed such that physiological 
signals modulate the effect that control of a task by other means, 
usually manual control, has on performance of the task. Such a 
modulation system enables a variety of human-human 
interactions based upon physiological self-regulation 

performance. These interpersonal interactions may be mixes of 
competition and cooperation for simulation training and/or 
videogame entertainment. 

Categories and Subject Descriptors 

H. 5.2 User Interfaces: Input devices and strategies; B.4.2 
Input/Output Devices: Channels and controllers. 

General Terms 

Design 

Keywords 

Physiological modulation, biofeedback 

I. INTRODUCTION 

The interpersonal biocybernetics concept is an extension of what 
may be termed an intrapersonal biocybernetics design. In the 
intrapersonal design, a participant’s own physiological signals 
are made to influence or modulate their manual control of an 
interface. The intrapersonal design has been implemented by 
modulating manual videogame controllers and wireless motion 
videogame controllers. 

A key characteristic of this design is the distinction between 
direct control and modulation of control. This distinction 
parallels the distinction made by Fairclough [2] between 
intentional brain-computer interface (BCI) operation and the 
biocybernetic adaptation paradigm that is based on spontaneous 
operator functional state. Physiological modulation of manual 
control differs from the conventional BCI practice of direct 
control of a task via physiological signals in an important way. A 
physiologically modulating system is a human-computer 
interaction system (HCI) designed such that physiological signals 
modulate the effect that control of a task by other means, usually 
manual control, has on task performance. In this way, 
physiological modulation integrates BCIs as intelligent sensors 
synergistically with the manual mode of control input. 
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2. INTRAPERSONAL MODULATION 

2.1 Software Modulation 

The earliest embodiment of the physiological modulation 
concept was implemented in game software in a “space battle” 
scenario [7]. In this modulated prototype, a “difficulty adjuster” 
was programmed to vary according to an 
electroencephalographic (EEG) band power ratio, termed an 
“engagement index” [8], In one mechanic, the subjective time of 
the game was correlated with the engagement index. Specifically, 
a delta time value was calculated based on the value of the EEG 
ratio. As this ratio value increased, the delta time, or dTime, 
decreased. This decreased value of dTime caused the subjective 
time of the game, i.e., the relative speed of target objects, to slow 
down, thus making it much easier for the player to hit targets and 
score points. The objective time of the game remained unaffected 
by the ratio level. Likewise, as the engagement index decreased, 
the relative speed of the target objects increased, thereby making 
the game more difficult and discouraging decreases in the 
engagement index. The player is thus encouraged to maintain a 
high ratio level since the game is easier and scores are improved. 
In addition to correlating the subjective time of the game with the 
ratio level, the prototype altered other parameters. For example, 
the normal operation of the selected video game introduced an 
effect termed jitter or evasiveness, which resulted in the position 
of the targets constantly changing in an unpredictable, seemingly 
erratic pattern, thereby increasing the difficulty of the game. This 
jitter effect was conventionally achieved by, e.g., calculating new 
X and Y positions for the graphic images using a randomly 
generated incremental change from the previous x and y values 
establishing position at a particular time. The EEG-modulated 
prototype inversely related the degree of this incremental change 
to the EEG band ratio level, i.e., the evasiveness decreased as the 
level increased and visa versa. In these ways, the player is 
encouraged to maintain high levels of the ratio index in order to 
succeed in playing the game. 

This approach has the disadvantage of requiring extensive 
reprogramming of a video game, or the complete construction of 
a new video game, in order to implement the modulation 
concept. Because much video game software is proprietary 
and/or not available in source code, this approach would not be 
usable for implementing the interpersonal biocybernetics 
concept. 

2.2 Hardware Modulation 

Another approach [6] involves modulation that transforms the 
controller signals received at the computer's port prior to their 
being used by the computer simulation or game software. The 
result can be that the magnitude of the effect of the game or 
simulation's input device (e.g., joystick, game pad, steering 
wheel) is modulated by the strength of the physiological 



signal(s). One implementation modulates the voltage powering a 
joystick axis in proportion to a physiological signal. By making 
the joystick's "control authority" proportional to the 
physiological signal, the player is encouraged to change the 
physiological signal according to a programmed criterion (e.g., 
increase, decrease, or maintain) in order to perform better at the 
game task. Another implementation pulse-modulates the button 
presses that a user makes on a controller. The frequency and/or 
width of the pulses is made proportional to the physiological 
signal, in effect, imposing a governor on the button’s effect on 
the game or simulation by adjusting the effective closure time of 
the buttons. 

A new type of controller allows a player to make inputs to a 
video game by moving the entire controller itself, thus requiring 
a different method of physiological modulation. One 
implementation exploits these motion-sensing technologies of the 
wireless Nintendo™ Wii™ videogame remote to modulate the 
player’s movement inputs to the videogame based upon 
physiological signals. Prototypes of one design [4, 11] enhance 
the challenge of various games by physiologically modulated 
attenuation (motion-control dampening) and/or disruption (cursor 
control perturbation). Games and simulations that are appropriate 
for this manipulation include first person shooter games using 
crosshairs position and point of view perturbations, medical 
simulations that use surgical instrument position perturbation 
based on autonomic measures reflecting anxiety to teach stress 
management during medical procedures, and sports simulations 
that adjust golf stroke strength based on electroencephalographic 
signals reflecting concentration. 

These modulation influences can emulate the amplification of 
nervousness when performing a task under pressure or the 
increase in the likelihood of error when becoming bored. Or, on 
the other hand, the enhancement of manual ability when 
performing a task “in the zone.” 

In one embodiment, the standard sensor bar of the Wii™ video 
game system is replaced with an array of infrared light-emitting 
diodes (LEDs) that are individually controllable on and off. This 
embodiment turns LEDs in the array on and off in dynamic 
patterns to produce a disturbance in the player-directed control of 
the infrared-sensing videogame controller, resulting in a 
disruption in the stability of the player-controlled object (e.g., a 
cursor) on the screen. The degree of the disruption is 
programmed to be proportional to the difference between the 
player's current momentary physiological signal value and a pre- 
selected target value. In this way, the player is rewarded for 
diminishing this difference by an improved ability to accurately 
position the player-controlled object (e.g., a cursor) on the 
screen. 

A microcontroller samples amplified and processed physiological 
signals (e.g., cardiotachometer readings), tests the signal levels 
with programmed conditional instructions, and, in accordance 
with the physiological signal condition determinations, adjusts 
the degree of the disruption. A similar scheme dynamically 
adjusts digitally controlled potentiometers that are configured as 
voltage dividers to modulate the voltage from the battery supply 
that powers the controller accelerometers and/or gyroscopes. 
This manipulation is intended to enhance the challenge of 
various games by physiologically modulated attenuation 
(dampening) of a user’s motion input (e.g., golf stroke), 
providing an incentive to produce the target physiological 
signal(s). 


3. INTERPERSONAL MODULATION 

Extending intrapersonal modulation to the interpersonal domain 
is intended to catalyze interaction among participants around 
physiological self-regulation performance challenge. These 
inteipersonal interactions may be mixes of competition and 
cooperation for simulation training and/or videogame 
entertainment. 

Physiological modulation technologies that operate by altering 
the functioning of game controllers, i.e., externally to game 
consoles, may be readily set up to allow players to interact with 
the game, and compete with each other, on a 

psychophysiological level, adding a new dimension to play - as 
well as expanding the skill set required. Competitive interaction 
with these intrapersonal modulation functionalities may be 
implemented with the standard multi-player capability of off-the- 
shelf game systems. 

However, a multiplayer format is also envisioned wherein 
physiological self-regulation skill contributes to success in ways 
analogous to traditional game-playing skills, i.e., scoring, power- 
ups, etc. 

3.1 Multi-User Modulation Design 

Norris [5] pointed out that: 

Top athletes [...] make excelling in their chosen type of 
psychophysiological control of the craniospinal system and 
striate muscles the major focus of their efforts. Here in the 
West, we value objectivism, material things, and have 
turned our attention outward toward controlling the external 
world. In the East, it has been quite different; the orientation 
of science and society values subjectivism and numinal 
experiences, and they have turned their attention inward, 
toward control of the internal world. Therefore, their 
champion ‘acrobats’ are those individuals who can assert 
and demonstrate control over autonomic and other internal 
processes. [...] in India, they have a sort of autonomic 
Olympics, where yogis and adepts come from far and wide 
to demonstrate their prowess at such things. . . 

In the present day, this sort of event could take place in a multi- 
player environment such as a videogame tournament. 

The design of a psychophysiologically competitive, as well as 
cooperative, environment is proposed in “Physiological Interface 
for a Multi-User Virtual Environment (MUVE) [9] intended to 
augment “a computer user's sense of immersion in the computer- 
generated multi-user virtual environment by adding physiological 
interactivity to the multi-user virtual environment and by 
influencing the scoring of performance in the multi-user virtual 
environment with measured values of a user's physiological 
functions” (e.g., an EEG-based engagement index [8]). 

The aim is to make any multi-user videogame or multi-user 
computer-simulated task more engaging and hence more 
effective as a device for teaching physiological self-regulation, 
by incorporating, as a competitive or cooperative interactive 
feature, each user's ability to self-regulate aspects of his or her 
physiology. 

This MUVE design is intended as an enhancement of multiplayer 
videogames that computes physiological scores standardized on 
an individual’s baseline responses - to level the playing field, and 
compares a selected score with a benchmark target or benchmark 
normative value. Depending on the results of the comparison, at 
least one game consequence is selected which may take various 



forms: a change in a user's point score; or a change in some 
element in the MUVE, i.e., a "power element" that is important 
for further success, such as the accrual of virtual fuel, food, 
ammunition, energy, longevity, shielding; or, a change in some 
virtual power, capability or function of a user's avatar that is 
important for further success, such as range of motion, or effect 
of motion, or duration of activity; or the user’s control of the 
MUVE control input device may be made more or less difficult. 
Based on the user’s perfo nuance, game consequences may also 
take the form of similar competitive or cooperative consequences 
for the user’s opponents or teammates. Each consequence may be 
made proportional to a functional combination of physiological 
comparison values. 

Additionally, a user may qualify for a gain of a certain 
advantageous element or game privilege only if the user has 
attained a threshold score that has been defined for that element 
or privilege. For example, upon attaining a threshold score, a 
player may selectably donate or withdraw performance points to 
or from the score of each other user interacting with the MUVE, 
adding an additional competitive or a cooperative aspect to the 
interaction. 

One component of a user’s performance is determined by each 
user's skill in using an input device to control the actions of his or 
her avatar in various scenarios and encounters arising within the 
MUVE, such as the outcome of combat with an enemy avatar in 
a videogame embodiment of a MUVE, or the outcome of a 
cooperative effort with a teammate avatar in a computer- 
simulated task embodiment of a MUVE. A second component is 
determined by each user's skill in self-regulating his or her own 
physiological functioning while controlling his or her avatar 
using an input device. The second component reflects the 
competition or cooperation engaged in by each user of the 
MUVE using measured values of his or her physiologic 
functions. 

A player’s physiological performance record accrued in the 
course of a session in the physiologically modulated MUVE may 
subsequently be accessed by software or another user. For 
example, the physiological performance record may 
subsequently be assigned to another "assignee" avatar whose 
behavior in the MUVE is thereafter determined and controlled by 
utilizing the player’s performance record, and the player's avatar 
may be activated while the player is not participating in the 
MUVE. The assignee avatar assumes the recorded physiological 
performance of the absent player, and emulates the player’s 
recorded physiological reactions to certain events occurring in 
the MUVE, such as parrying a sword thrust, thereby effectively 
serving as a "ghost" of the player with which other users of the 
MUVE may interact in the absence of the player’s actual 
participation in the MUVE. 

3.2 Collaborative Modulation Design 

The entertainment value and social interaction experience of 
electronic gameplay may be enhanced by distributing the control 
and modulation of inputs to electronic games among two or more 
players, so that joint game goals are collaboratively pursued and 
accomplished by separate players who provide different means of 
control and modulation [10]. For example, one or more players 
may provide physical activity control via game controller(s) 
(these players may be termed the physical operators) and one or 
more other individuals influence the game through physiological 
activity measured via body sensors (these players may be termed 
physiological operators). When applied to neurofeedback 


training, such collaboration encourages the therapist or parent to 
provide social support by engaging in a training session as the 
physical operator, while the trainee practices self-regulation as 
the physiological operator. In a training simulation context, 
collaboration would consist of a physiological operator (e.g., the 
pilot-monitoring) practicing a psychophysiological state 

associated with sustained attention while the physical operator 
(e.g., pilot-flying) practices manual operation. This 

interdependent mode of operation could provide novel material 
for the communication aspect of crew resource management 
(CRM) training [3], 

Some physiologically modulated game designs [6, 7, 11] lend 
themselves readily to collaborative interaction by having 
different players assume the physical and physiological operator 
roles with the same modulated controller [10]. Such an 
embodiment avoids the problem of movement disruption of 
physiological sensing by modulating one player's game controller 
using the physiological signals of another, collaborating player 
who is physically inactive. This functionality further enables 
collaborative team play by multiple players with different roles 
on the team - one or more providing physiological mastery to 
facilitate game performance and one or more others 
simultaneously providing manual or physical skills needed to 
operate the game. This creates richer and more complex gaming 
opportunities than present games provide, enabling engaging and 
rewarding team social interactions among people who have 
different skill sets and interests or who take turns playing 
different roles on a team - either providing the physiological self- 
mastery skill or providing physical performance action skills. 

This design modulates one or more players' game controller (i.e., 
game input device operated by physical activity) using the 
physiological signals of another/other collaborating player(s) 
who is/are not providing input to the game via a game controller. 
This function is accomplished by transmitting control signals, 
either wired or wirelessly, derived from one or more players’ 
physiological signals to modulate the control over the game of 
one or more other players who are using electronic game 
controllers, in a way that either limits or boosts the game 
performance influence of the game controllers. 

Alternatively, this shared influence over a game of a 
physiological operator and physical (game controller) operators 
can be implemented by blending influences from both into a 
single control input into the game; in this manner, game action 
tasks could be operated either via a game controller or particular 
predefined physiological input, or through pre-defined 
combinations of both. For instance, a video game task of an 
avatar game character jumping over a canyon could be initiated 
by either the game controller operator or by a physiological 
operator's particular body activity, and the height or length of the 
jump could then be solely dependent on the physiological 
activity of the physiological operator whereas the direction of the 
jump and place of landing on the other side could be controllable 
only by the game controller input. Thus, the physiological and 
game controller operators would have to collaborate to succeed 
at the task of jumping for success in the game. 

Two new categories of electronic game play are made possible: 
1) modulating one or more player’s game controller input using 
the physiological signals of another/other collaborating 
individuals; and 2) modulating game activity through joint 
authority over game actions by game controller input from one or 
more individuals and physiological sensor input from other 
separate individuals not using game controllers. Physiological 



and manual control of electronic games can be allocated in 
various combinations to several players. 

Individuals who are physically challenged may participate in 
electronic game play by collaborating with a player who is able 
to manipulate controls that the challenged player cannot, and 
enables individuals with different skill sets and interests 
(physiological self-control vs. physical performance skills) to 
join together in rewarding game play. 

Unprecedented electronic games can be created that incorporate a 
classic motif in superhero comics and movies in which 
differently-abled protagonists collaborate and/or interact, such as 
Professor X and his cohorts in the X-Men series, and provides 
opportunities for new kinds of games where the application of 
"mental powers" of particular individuals (e.g., particular 
deliberately produced EEG characteristics) can enable or enrich 
the performance of other individuals playing a game. 

4. DISCUSSION 

Certain human physiological states are associated with the 
optimal performance of certain activities and behaviors, or the 
attainment of superior states of health. Biofeedback training 
directed to self-regulating such physiological states seeks to 
optimize the performance of their associated actions, behaviors, 
and states of health, thereby facilitating mastery in action and well- 
being. For example, a self-induced reduction of an elevated heart 
rate and blood pressure, a rise typically associated with anxiety, 
may be expected to move a person into a state of superior focus 
with respect to a challenge and its resolution - a state that is better 
suited for victory over an opponent in real or simulated combat or 
completion of a real or simulated task [1], 

Biofeedback training techniques for physiological self-regulation 
are fundamentally grounded upon repetition and the frequency of 
repetition, techniques for which persons have varying and limiting 
tolerances, particularly if their practice of biofeedback is 
undertaken in an environment that is not motivating because it fails 
to engender excitement, enthusiasm, challenge, stimulation, or 
curiosity. The absence of such motivating feelings can result in 
nonadherence to biofeedback training - a persistent and pervasive 
problem that has limited the prevalence and effectiveness of 
biofeedback training. 

However, biofeedback training programmed into videogames or 
computer simulations has been limited to single-user videogames 
or single-user computer simulations, where the user is made aware 
of only some of his or her own monitored physiological functions, 
and the user's interaction with the videogame or computer 
simulation is modulated only by the control of his or her own 
physiological functions [6, 7, 11], In effect, the user competes only 
with himself or herself, and this may limit commitment to play and 
participation, with a concomitant decline in biofeedback-mediated 
self-regulation of the user's physiological state. 

Recent biofeedback technology innovations have solved this 
problem to a degree, by blending biofeedback into videogames, 
and thereby making training inherently motivating. 

On the other hand, physiologically modulated videogame or 
computer simulation systems also enable a variety of human- 
human interactions based upon physiological self-regulation 
performance. These interpersonal interactions may be mixes of 
competition and cooperation for simulation training and/or 
videogame entertainment. 


Physiologically modulating games and simulations has the effect 
of projecting exaggerated effects of the physiological 
concomitants of emotion and cognition into the interaction. 
Incorporating social interactivity into physiologically modulated 
games and simulations further engages users in human-computer 
interaction experiences that have the capacity to train valuable 
mental skills beyond hand-eye coordination. 

When physiological signal(s) are the target of physiological self- 
regulation or biofeedback training, modulated game play 
reinforces therapeutic changes in related physiological processes. 
However, the reinforcing feedback is implicit in the game play 
and not explicit in the form of direct feedback (bar graphs, 
tracings), as in conventional biofeedback training. In this way, 
contingencies for subtle conditioning of the desirable 
physiological response(s) are set up. 

The added benefit of the physiologically modulated approach is 
the improvement of self-regulation capabilities that directly 
impact a player’s health in a positive manner. Videogames and 
simulations can be designed such that success in the virtual world 
is predicated upon self-regulation skills demonstrative of healthy 
autonomic function and optimal cognitive states. Evoking certain 
psychophysiological states is a design choice that developers are 
currently making de facto, but with the result that games and 
simulations are often devices that produce frustration and foster 
poor attention abilities. The decision by designers to incorporate 
physiological modulation can be considered a next step in a more 
informed and healthy direction for gaming and simulation 
products. Making this choice allows users to choose technology 
that is entertaining as well as beneficial to their physical, mental 
and interpersonal health. 

The approaches to physiological computing described in this 
paper represent a unique blending of conventional computer 
interface capabilities with contemporary psychophysiological 
monitoring and biofeedback techniques. The use of 
physiologically modulated videogames and simulations 
involving multiple players using traditional manual control and 
BCIs as intelligent sensors promises to augment HCI and human- 
human interaction (HHI) applications. Tapping into the natural 
social experience that occurs when players compete or 
collaborate in gaming or simulated environments leverages 
subtle interpersonal interactions. The addition of the challenge of 
physiological self-regulation in a social context improves the 
motivation to perform in order to enable team success or victory 
over opponents in simulated environments. Further, 
implementing this challenge in the ways described herein fosters 
dialogue around self-regulation among the participants. These 
implementations can require dividing attention between one’s 
own self-regulation performance and that of others, a potentially 
valuable skill for team settings. 

How an interface can adapt to the user to provide support is an 
important consideration for BCI research. Current BCI 
technology has specific application and usability goals at the 
fore. Establishing reliable capabilities is of foremost importance, 
but, beyond utility, these systems can be designed to inherit the 
benefit of social experience that occurs between humans. The 
application of readily available physiological information from 
BCI technology can be used to enable such a user experience. 
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